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Background / Context: 

Description of prior research and its intellectual context. 

For many years, educators have been concerned about the low level of mathematics 
performance of U.S. students in relation to national standards and in international comparisons 
(National Research Council, 2001). Difficulties in mathematics achievement are particularly 
severe for students from low income and minority backgrounds (National Assessment of 
Educational Progress, 2007) and warning signs of problems appear early. Significant differences 
between subpopulations of students can be reliably measured at school entry on measures related 
to counting principles and number knowledge to more complex understandings of quantities, 
operations, and problem solving (Griffin, Case, & Siegler, 1994; Jordan, Kaplan, Olah, & 
Locuniak, 2006). A number of studies (Bodovski & Farkas, 2007; Duncan et al., 2007; Morgan, 
Farkas, & Wu, 2009) have documented the relationship between early and later math 
achievement (e.g., kindergarten to fourth grade). Using a nationally representative sample of 
students from the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS- 
K), Morgan et al. (2009) found that students who were in the lowest 10 th percentile at both the 
beginning and end of kindergarten had a 70% chance of remaining in the lowest 10 th percentile 
five years later. One promising approach to improving kindergarten and later math achievement 
is delivering effective instructional programs to all students when they enter school to bridge the 
emerging achievement gap and promote early mathematics learning. 

Effective instructional and intervention programs for the full range of students in 
classrooms — from students who are on track for successful learning outcomes to students who 
are experiencing difficulty and need additional service and support — is the basis of a Response to 
Intervention (RTI) service delivery model. RTI is an integrated approach that spans general, 
remedial, and special education (Fuchs, Fuchs, & Compton, 2004). Rtl emphasizes high quality 
instruction in general education, interventions for at-risk students that are matched to 
instructional need, and the use of data to document student responsiveness to instruction and 
intervention for the purpose of making ongoing educational and instructional decisions (Batsche 
et al., 2005). Although variations to Rtl models exist (Gersten et al., 2009; NASDSE, 2006) most 
tiered models have 3 levels of support with Tier 1 consisting of the instruction delivered in 
general education by the classroom teacher. Tier 2 is targeted instruction delivered in small 
groups to at-risk students and Tier 3 is provided to students who have not responded to Tier 2 
small group interventions. Tier 3 typically consists of highly intense interventions including one- 
on-one tutoring. 

The purpose of this study, funded by the Institute for Education Sciences under the 
Mathematics and Science Education topic, was to test the efficacy of a Tier 2 intervention 
curriculum, ROOTS , when implemented under rigorous experimental conditions. In this study all 
classrooms implemented a validated Tier 1 core curriculum (Clarke et al., in press), and 
classrooms were randomly assigned to treatment and comparison conditions, blocking on school. 
Treatment classrooms implemented the validated Tier 1 core curriculum plus the ROOTS an 
intervention program for at-risk students. Comparison classrooms implemented the validated 
Tier 1 core curriculum plus standard district practice for intervening with at-risk students. 

The validated Tier 1 curriculum used by all classrooms was the Early Learning in 
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Mathematics curriculum (ELM). As part of IES funding for this project, in a previous study 
(Study 1) ELM was tested in a randomized controlled trial, with treatment classroom 
implementing ELM as their Tier 1 mathematics curriculum and comparison classrooms 
implementing the district’s standard mathematics curriculum for Tier 1 (Clarke, Smolkowski, 
Baker, Fien, & Chard, in press). ELM had a statistically significantly impact on the mathematics 
achievement of kindergarten students overall (p = .03; Hedges’ g = 0.14) and it was particularly 
beneficial for at-risk students (p < .01; g = 0.24), although at-risk students did not catch up to 
their average-achieving peers by the end of the study (Clarke et ah, in press). The content of the 
year-long ELM curriculum focuses on (a) whole number and operations; (b) measurement; (c) 
geometry; and (d) precise mathematics vocabulary. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

This presentation will address our primary intervention research question for Study 2: For 
at-risk students, what is the impact on mathematics achievement of the ROOTS intervention 
delivered in conjunction with an effective core curriculum (ELM) compared to an effective core 
curriculum (ELM) and standard district practice for intervening with at-risk students? This 
comparison holds constant the core curriculum, allowing us to focus of the intervention with at- 
risk students. We expect that classrooms that implement ELM plus ROOTS for at-risk students 
will have higher proximal and distal achievement scores than classrooms that implement ELM 
plus standard district intervention practice for at-risk students. 



Setting: 

Description of the research location. 

Study 2 was conducted in Oregon during the 2009-2010 school year. The study took 
place in twelve schools in three medium sized (12,062-20,714) suburban school districts located 
in the northwest. The school districts had similar demographics with a median of 12.2% 
receiving special education services, 1 1.2% are English language learners, and 44.9% are eligible 
for free/reduced lunch. A total of 29 kindergarten classrooms were randomly assigned to 
treatment and comparison conditions. The amount of time mathematics instruction was provided 
each day for at-risk students was the same in treatment and comparison classrooms. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

The study sample included two participant groups: kindergarten instructional assistants 
and at-risk students in the participating kindergarten classrooms. The instructional assistant 
sample included 29 different instructional assistants. The student sample included 143 students. 
To be eligible for ROOTS, student had to meet two criteria. First, they had to score below the 
40 th percentile on the TEMA at pre-test. Second, teachers were provided a list of all students in 
their classroom that met the TEMA criteria. Teachers then nominated five students to receive 
ROOTS who they believed would most benefit from a Tier 2 intervention. In comparison 
classrooms all students were pretested on the TEMA and teachers identified the students below 
the 40 th percentile on the TEMA who they believed would benefit most from a Tier 2 
intervention 
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Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

For Track 2, this may include the development and validation of a measurement instrument. 

ROOTS is a 50-lesson Tier 2 intervention program designed for students who are not 
responding to validated Tier 1 instruction. Instructional assistants (IAs) implement ROOTS, 
teaching two to three lessons per week, for approximately 20 weeks. 

ROOTS was designed according to instructional design principles shown to effective for 
at-risk students (Gersten et ah, 2009; National Mathematics Advisory Panel, 2008). ROOTS 
focuses exclusively on the development of whole number understanding. Specific focus areas are 
developing proficiency in numeration to 20, counting, identifying and writing numbers to 20, 
counting from a number other than 1, reading and solving simple addition statements, 
understanding place value and model numbers to 20, and identifying quantities and numbers that 
are more, less, and equal. 

Research Design: 

Description of the research design. 

Study 2 evaluated ROOTS within a cluster-randomized controlled trial (Murray, 1998, 
2001). The study nested students within classrooms and classrooms nested within conditions, and 
classrooms were randomly assigned to condition within schools. Math achievement data were 
collected from individual students, and random assignment and instructional delivery took place 
at the classroom level. Project staff tested students on math achievement at the beginning (Ti) 
and end (T 2 ) of the students’ kindergarten school year with the Test of Early Mathematics 
Ability-Third Edition (Ginsburg & Baroody, 1990) and a set of early numeracy curriculum-based 
measures (Clarke, Baker, Smolkowski, & Chard, 2008). 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

For Track 2, this may include the use of existing datasets. 

Trained staff members collected all student and classroom observation data after meeting 
reliability standards for each measure. The primary outcome was performance on the TEMA-3. 
The TEMA-3 is a norm-referenced individually administered measure of early mathematics for 
children ages 3 to 8 years 1 1 months. The TEMA-3 measures both formal and informal 
mathematics including skills related to counting, number facts and calculations, and related 
mathematical concepts. The alternate-form reliability of the TEMA-3 is reported as .97 and test- 
retest reliability ranges are reported to be from .82 to .93. Concurrent validity with other criterion 
measures of mathematics is reported as ranging from .54 to .91. Secondary outcome measures 
were Early Numeracy-Curriculum-Based Measures (EN-CBM) (Clarke & Shinn, 2004). EN- 
CBM are a set of measures based on principles of curriculum-based measurement (Shinn, 1989). 
Each 1 -minute fluency-based measure assesses an important aspect of early numeracy 
development including oral counting, magnitude comparison and strategic counting. Concurrent 
and predictive validity range from .46 to .72 with other published standardized measures. 

We assessed intervention effects on each of the primary outcomes with a nested time by 
condition analysis (Murray, 1998). This analysis approach tests differences between conditions 
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on change in outcomes from the beginning of kindergarten (Ti) to the end of kindergarten (To). 
This analysis approach included all data — whether or not a student’s scores were present at both 
time points — to estimate differences between assessments times and between conditions. It also 
accounts for autocorrelation among assessments within individual students and the intraclass 
correlation associated with multiple students nested within the same schools. As a test of net 
differences, it provides a straightforward interpretation of the results. 

We computed an effect size, Hedges’ g (Hedges, 1981), for each fixed effects. Hedges’ g 
represents an individual-level effect size comparable to Cohen’s d (Rosenthal, Rosnow, & 

Rubin, 2000); Cohen’s d uses a sample standard deviation and Hedges’ g uses a population 
standard deviation (Rosenthal & Rosnow, 2008). Hedges’ g is recommended by the What Works 
Clearinghouse (2008) for multilevel models and characterizes the total effect, which ignores 
clustering. It tends to be more conservative than other effect size indicators. 



Findings / Results: 

Description of the main findings with specific details. 

The results of the analyses suggest that at-risk students may have performed better in 
classrooms that provided ROOTS than in classrooms that offered standard district practice. We 
found small to medium effect sizes for the TEMA (g = 0.32), CBM total (g = 0.31), and one 
individual CBM measure, Quantity Discrimination (g = 0.50). All effect sizes meet the WWC 
standards (What Works Clearinghouse, 2008) for showing a “substantively important positive 
effect” (p. 54). Although the effect sizes were encouraging, we were unable to reject the null 
hypothesis for the TEMA (p = .066) or CBM total (p = .144). For Quantity Discrimination, 
however, we found a statistically significant result (p = .01 1). All tests used 27 degrees of 
freedom corresponding to the classroom level. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

This presentation will cover the findings of a cluster-randomized controlled evaluation of 
the ROOTS Tier 2 intervention. ROOTS was implemented within the context of an empirically 
validated core curriculum. We will address the issue of having sufficient power in this study to 
detect statistically significant effects and we will describe how a second wave of data will be 
collected to improve overall power for Study 2. We will discuss possible implications of our 
findings in the context of response to intervention (RTI) service delivery models. The findings 
from this study will be used to illustrate how research-based curricula can be used at multiple 
tiers to address the learning needs of at-risk students and reduce the achievement gap between at- 
risk students and their average-achieving peers. 
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